A Parallel PARAFAC Implementation & Scalability Testing for Large-Scale Dense Tensor Decomposition

Parallel Factor Analysis (PARAFAC) is used in many scientific disciplines to decompose multidimensional datasets into principal factors in order to uncover relationships in the data. While quite popular, the common implementations of PARAFAC are single server solutions that do not scale well to very large datasets. To address this limitation, a Parallel PARAFAC algorithm has been designed and implemented in C using MPI. The end-to-end pipeline includes a parallel read of the input data from a file, the execution of the parallel algorithm, and concludes with a parallel write of the results to a file. The implementation has been evaluated using a strong scaling study on an IBM Blue Gene/Q supercomputer. The compute time, as well as the communication, file read, and file write bandwidths were each captured across multiple scenarios to evaluate the overall system performance and scalability. Results indicate the implementation scales well—with a 128x increase in the number of parallel processes, the system executed 200x faster. Further, the communication time at its peak accounted for only 12% of the total processing time, indicating the implementation is currently CPU bound and thus should continue to scale well across more and more nodes.